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(54) TiUe: METHOD AND APPARATUS FOR ADAPTIVE FILTER TAP SELE(mON ACCORDING TO A CLASS 



(57) Abstract 

A method, apparatus, and article of man- 
ufacture for restoring a deteriorated signal (201) 
to an undeteriorated signal (213), A deteriorated 
signal (201) consists of a plurality of deterio- 
rated and undeteriorated data points. For each 
deteriomted data point, a plurality of class types 
is created based upon characheristics of the area 
containing the deteriorated data point. The data 
point is classified with respect to one of the plu- 
rality of class types and assigned a corresponding 
input signal class (217). The undeteriorated sig- 
nal is generated by adaptively filtering the dete- 
riorated input signal in accordance with the input 
signal classification result More than one classi- 
fication method is used to create the plurality of 
class types (219). Created classes may include 
a motion class (1809), an error class (1811), a 
spatial class or a spatial activity class. The filter 
taps (1813) are selected adaptively according to 
tiie plurality of class types. The filter taps may 
be selected adaptively according to the motion 
and error class (1813). 
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METHOD AND APPARATUS FOR ADAPTIVE FILTER TAP SELECTION 

ACCORDING TO A CLASS 

FIELD OF THE INVENTION 

This invention relates generally to the processing of image, sound or other 
correlated signals, and more particularly, to a method, apparatus, and article of 
manufacture for restoring a deteriorated signal to an undeteriorated signal. 

BACKGROUND OF THE INVENTION 

Conventionally, to restore an image that is deteriorated in image quality it is 
necessary to analyze the cause of the deterioration, determine a deterioration model 
function, and apply its inverse function to the deteriorated image. Various causes of 
deteriorations are possible, such as a uniform movement of a camera (imaging device 
such as a video camera) and blurring caused by the optical system of a camera. 
Therefore, in restoring an image, different model functions may be used for respective 
causes of deteriorations. Unless the cause of deterioration is found, it is difficult to 
restore a deteriorated image because a model function cannot be determined. 

In addition, it is frequently the case that even if a model function of a 
deterioration is established, there is no inverse function for restoration that corresponds 
to the model function. In such a case, it is difficult to perform evaluation for 
determining the optimum model. 

Conventionally, error recovery has been achieved by correlation evaluation. 
For example, some recovery choices have been implemented using a conventional error 
pixel recovery method. FIG. 1 A shows a conventional error recovery block diagram. 
Using neighboring data, which are shown in FIG. IB, spatial inclinations of the target 
data are detected. In this example, the inclinations regarding four directions are 
evaluated according to the formulae which are shown in FIG. IC. An interpolation 
filter is chosen where the inclination value, Eu is the smallest among four values. In 
addition to the spatial inclination, a motion factor is also evaluated for error recovery. 
In the case of the motion area, a selected spatial filter is used for error recovery.. On the 
other hand, the previous firame data at the same location as the target data are used for 
error recovery. This evaluation is performed in the evaluation block of FIG. lA. 

The conventional error recovery process shown in FIGs. lA-lC may cause 
many serious degradations on changing data, especially on object edges. Actual signal 
distribution typically varies widely, so these problems are likely to occur. Therefore, 
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there is a need for a way to restore a deteriorated signal to an undeteriorated signal 
which minimizes degradations on changing data. 

SUMMARY OF THE INVENTION 

The present invention provides a method, apparatus, and article of manufacture 
for restoring a deteriorated signal to an undeteriorated signal. A deteriorated signal 
consists of a plurality of deteriorated and undeteriorated data points. For each 
deteriorated data point, a plurality of class types is created based upon characteristics of 
the area containing the deteriorated data point. The data point is classified with respect 
to one of the plurality of class types and assigned a corresponding input signal class. 
The undeteriorated signal is generated by adaptive filtering of the input signal in 
accordance with the input signal classification results. More than one classification 
method may optionally be used to create the plurality of class types. Created classes 
may include a motion class, an error class, a spatial class or a spatial activity class. An 
adaptive class tap structure may optionally be used to create the plurality of class types. 
An adaptive filter tap structure may optionally be used base on the corresponding 
plurality of class types. Filter tap expansion may optionally be used to reduce the 
number of filter coefficients. The deteriorated input signal may optionally be modified 
by preprocessing peripheral erroneous data. A spatial class may optionally be modified 
according to spatial symmetry. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and may be better 
understood by referring to the following description in conjunction with the 
accompanying drawings, in which like references indicate similar elements and in 
which: 

FIGs. lA-lC show a conventional error recovery method, filter tap, and 
correspondence between inclination value and interpolation filter; 

FIGs. 2A-2D show a classified adaptive error recovery method and class 
compatible with an embodiment of the present invention; 

FIG. 3 shows a motion class tap compatible with an embodiment of the present 
invention; 

FIG. 4 shows an error class tap compatible with an embodiment of the present 
invention; 



wo 00/48117 



-3- 



PCT/USOO/03636 



FIG, 5 shows an adaptive spatial class tap compatible with an embodiment of 
the present invention; 

FIG. 6 shows an adaptive spatial class tap (error class 0) compatible with an 
embodiment of the present invention; 

FIG. 7 shows an adaptive spatial class tap (error class 1) compatible with an 
embodiment of the present invention; 

FIG. 8 shows an adaptive spatial class tap (error class 2) compatible with an 
embodiment of the present invention; 

FIG. 9 shows an adaptive spatial class tap (error class 3) compatible with an 
embodiment of the present invention; 

FIG. 10 shows an adaptive filter tap compatible with an embodiment of the 
present invention; 

FIG. 11 shows a motion class adaptive filter tap compatible with an 
embodiment of the present invention; 

FIG. 12 shows a motion class adaptive filter tap (error class 0) compatible with 
an embodiment of the present invention; 

FIG. 13 shows a motion class adaptive filter tap (error class 1) compatible with 
an embodiment of the present invention; 

FIG. 14 shows a motion class adaptive filter tap (error class 2) compatible with 
an embodiment of the present invention; 

FIG. 15 shows a motion class adaptive filter tap (error class 3) compatible with 
an embodiment of the present invention; 

FIG. 16 shows a preprocessing algorithm compatible with an embodiment of the 
present invention; 

FIG. 17 shows a motion tap and stationary tap preprocessing algorithm 
compatible with an embodiment of the present invention; 

FIG. 18 shows a system block diagram compatible with an embodiment of the 
present invention; 

FIG. 19 shows coefficient memory contents compatible with an embodiment of 
the present invention; 

FIG. 20 shows an ADRC class reduction based on a 4-tap 1-bit ADRC 
compatible with an embodiment of the present invention; and 

FIG. 21 shows an example of audio signal adaptive classification compatible 
with an embodiment of the present invention. 
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DETAILED DESCRIPTION OF AN EMBODIMENT 
OF THE PRESENT INVENTION 

In the following description of an embodiment of the present invention, 
reference is made to the accompanying drawings which form a part hereof, and in 
which is shown by way of illustration a specific embodiment in which the invention 
may be practiced. It is to be understood that other embodiments may be utilized and 
structural changes may be made without departing from the scope of the present 
invention. 

The present invention provides a method, apparatus, and article of manufacture 
for restoring a deteriorated signal to an undeteriorated signal using classified adaptive 
error recovery. Target data is the particular data of the deteriorated signal whose value 
is to be determined or estimated. 

Classified adaptive error recovery is the technology which utilizes classified 
adaptive filter processing. A proper classification with respect to the deteriorated input 
signal is performed according to the input signal characteristics. An adaptive filter is 
prepared for each class prior to error recovery processing. 

More than one classification method may optionally be used to generate the 
plurality of classes. Generated classes may include a motion class, an error class, a 
spatial activity class or a spatial class. An adaptive class tap structure may optionally 
be used to generate the plurality of classes. An adaptive filter tap structure may 
optionally be used according to the class which is detected in each deteriorated input 
signal. The adaptive filter tap structure may optionally be expanded based upon 
multiple taps. The number of filter coefficients that must be stored can be reduced by 
allocating the same coefficient to multiple taps. This process is referred to as filter tap 
expansion. The deteriorated input signal may optionally be modified by preprocessing 
peripheral erroneous data. A spatial class may optionally be eliminated according to a 
spatial class elimination formula. 

The present invention can be applied to any form of correlated data, including 
without limitation photographs or other two-dimensional static images, holograms, or 
other three-dimensional static images, video or other two-dimensional moving images, 
three-dimensional moving images, a monaural sound stream, or sound separated into a 
number of spatially related streams, such as stereo. In the description, the term value, 
in one embodiment, may refer to a component within a set of received or generated 
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data. Furthermore, a data point is a position, place, instance, location or range within 
data. 

For the sake of clarity, some of the description herein focuses on video data 
comprising a pixel stream. However, it will be recognized that the present invention 
may be used with other types of data other than video data and that the terms and 
phrases used herein to describe the present invention cover a broad range of 
applications and data types. For example, an adaptive class tap structure is an adaptive 
structure for class tap definition used in multiple classification, A spatial class, a 
motion class and an error class may be used to define the structure. An adaptive filter 
tap structure is an adaptive structure for filter tap definition based upon a corresponding 
class. 

A class may be defined based on one or more characteristics of the target data. 
For example, a class may also be defined based on one or more characteristics of the 
group containing the target data. A class ID is a specific value within the class that is 
used to describe and differentiate the target data fi-om other data with respect to a 
particular characteristic. A class ID may be represented by a number, a symbol, or a 
code within a defined range. A parameter may be used as a predetermined or variable 
quantity that is used in evaluating, estimating, or classifying the data. For example, the 
particular motion class ID of a target data can be determined by comparing the level of 
motion quantity in the block containing the target data against a parameter which can 
be a pre-determined threshold. 

Multiple Classification 

In one embodiment, a multiple class may be used as a collection of specific 
values or sets of values used to describe at least two different characteristics of the 
target data. For example, a multiple class may be defined to be a combination of at 
least two different classes. For example, a multiple class may be defined to be a 
combination of an error class, a motion class, and a spatial class such as an ADRC 
class. 

In one embodiment, the multiple class ID can be used as the memory address to 
locate the proper filter coefficients and other information that are used to determine or 
estimate the value of the target data. In one embodiment, a simple concatenation of 
different class IDs in the multiple class ID is used as the memory address. 
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Therefore, a multiple classification scheme is a way of classifying the target 
data with respect to more than one characteristic of the target data in order to more 
accurately detemnine or estimate the value of the target data. 

An error class is a collection of specific values used to describe the various 
distribution patterns of erroneous data in the neighborhood of the target data. In one 
embodiment, an error class is defined to indicate which adjacent data to the target data 
is erroneous. An error class ID is a specific value within the error class used to 
describe a particular distribution pattern of erroneous data in the neighborhood of the 
target data. For example, an error class ID of "0" may be defined to indicate that there 
is no erroneous data to the left and to the right of the target data; an error class ID of 
"1" may be defined to indicate that the data to the left of the target data is erroneous, 
etc. A filter is a mathematical process, function or mask for selecting a group of data. 

A motion class is a collection of specific values used to describe the motion 
characteristic of the target data. In one embodiment, the motion class is defined based 
on the different levels of motion of the block containing the target data, for example, no 
motion in the block, little motion in the block, or large motion in the block. A motion 
class ID is a specific value within the motion class used to indicate a particular level of 
motion quantity of the target data. For example, motion class ID of "0" may be defined 
to indicate no motion, motion class ID of "3" may be defined to indicate large motion. 

A spatial class is a collection of specific values used to describe the spatial 
characteristic of the target data. For example, spatial classification of the data may be 
determined using Adaptive Dynamic Range Coding (ADRC), Differential Pulse Code 
Modulation (DPCM), Vector Quantization (VQ), Discrete Cosine Transform (DCT), 
etc. A spatial class ID is a specific value within the spatial class used to describe the 
spatial pattern of the target data in the group or block containing the target data. 

For example, an ADRC class is a spatial class defined by the Adaptive Dynamic 
Range Coding method. An ADRC class ID is a specific value within the ADRC class 
used to describe the spatial pattern of the data distribution in the group or block 
containing the target data. A class is a collection of specific values used to describe 
certain characteristics of the target data. A variety of different types of classes exist, 
for example, a motion class, a spatial class, an error class, a spatial activity class, etc. 

The present invention provides a method and apparatus for adaptive processing 
that generates data corresponding to a set of one or more data classes. This process is 
known as "classification". Classification can be achieved by various attributes of 
signal distribution. For example. Adaptive Dynamic Range Coding (ADRC) may be 
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used for generation of each class as a spatial class, but it will be recognized by one of 
ordinary skill in the art that other classes, including a motion class, an error class, and a 
spatial activity class may be used with the present invention without loss of generality. 
A spatial activity class is a collection of specific values used to describe the spatial 
activity characteristic of the target data. For example, spatial activity classification of 
the data may be determined using the dynamic range, the standard deviation, the 
Laplacian value or the spatial gradient value. Some classification methods provide 
advantages which are desirable before restoration of a deteriorated signal takes place. 
For example, ADRC can achieve classification by normalizing each signal waveform 
automatically. 

For each class, a suitable filter for signal restoration is prepared for the adaptive 
processing. In one embodiment, each filter is represented by a matrix of filter 
coefficients which are applied to the data. The filter coefficients can be generated by a 
training process, an example of which is described subsequently, that occurs as a 
preparation process prior to filtering. In one embodiment of the present invention, the 
filter coefficients can be stored in a random access memory (RAM), shown in FIG. 2A 
at 207. 

A typical signal processing flow of the present invention is shown in FIG. 2A. 
Target input data 201 can be accompanied with error flag data 203. Error flag data can 
indicate locations within the data that contain erroneous pixels. In one embodiment of 
the present invention, an ADRC class is generated for each input target data in 
classification block 205, filter coefficients corresponding to each class ID are output 
from the coefficient memory block 207, and filtering is executed with input data 201 
and the filter coefficients in the filter block 209. The filtered data may correspond to an 
error recovered result. In the selector block 211, switching between error recovered 
data and error firee data occurs according to the error flag data 203. 

In FIG. 2B, an example is shown where the number of class taps is four. In the 
case of 1-bit ADRC, 16 class IDs are available as given by [formula 3], shown below. 
ADRC is realized by [formula 2], shown below. Detecting a local dynamic range (DR) 
is given by [formula 1], shown below. 



DR^MAX-MIN + \ 



[formula 1] 




[formula 2] 



V 
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4 



[formula 3] 



where c corresponds to an ADRC class ID, DR represents the dynamic range of the 
four data area, MAX represents the maximum level of the four data, MIN represents the 
minimum level of the four data, qi is the ADRC encoded data, also referred to as a Q 
code, and Q is the number of quantization bits. The L J operator represents a 
truncation operation. 

In 1-bit ADRC, c may have a value from 0 to 15 with Q = 1. This process is 
one type of spatial classification, but it will be recognized by one of ordinary skill in 
the art that other examples of spatial classification, including Differential PCM, Vector 
Quantization and Discrete Cosine Transform may be used with the present invention 
without loss of generality. Any method may be used if it can classify a target data 
distribution. 

In the example shown in FIG. 2C, each adaptive filter has 12 taps. Output data 
is generated according to the linear combination operation given by [formula 4], shown 
below, 



where x/ is input data, corresponds to each filter coefficient, andy is the output data 
after error recovery. Filter coefficients can be generated for each class ID by a training 
process that occurs prior to the error recovery process. 

For example, training may be achieved according to the following criterion. 




[formula 4] 




[formula 5] 



where and Y are, for example, the following matrices: X\s the input data matrix 
defined by [formula 6], Wis the coefficient matrix defined by [formula 7], and Y 
corresponds to the target data matrix defined by [formula 8], 
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X = 



t2i ^22 



[formula 6] 



ml ■*»l2 



W2 



[formula 7] 



Y = 



f \ 

\yj 



[formula 8] 



The coefficient Wj can be obtained according to [formula 5], so that estimation 
errors against target data are minimized. 

In the example shown in FIG. 2C, 12 coefficients regarding each ADRC class 
ID are determined by the training method described above. 

A flow diagram of an embodiment of the present invention is shown in FIG. 2D. 
The flow chart of FIG. 2D shows the basic processing stream for generating an 
undeteriorated signal from the deteriorated input signal. At step 215, the preprocessing 
for a peripheral erroneous pixel is performed. At step 217, each classification 
regarding the deteriorated input signal is executed to generate a class ID. Some class 
taps are selected adaptively according to another class ID. Multiple classification may 
be executed, such as motion classification, error classification, spatial activity 
classification and spatial classification. 

The classification scheme can be defined during system design, where the 
classification scheme, the number of classes, and other specification are decided for the 
target data. The design stage may include, among others, considerations of system 
performance and hardware complexity. 

At step 219, multiple classification generates a multiple class ID with a plurality 
of class IDs which are generated by various classification at step 217. At step 221, 
filter taps are adaptively selected according to the multiple class ID which is generated 
at step 219. At step 223, the filter tap structure is adaptively expanded according to the 
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multiple class ID which is generated at step 219. The number of filter coefficients that 
must be stored can be reduced by allocating the same coefficient to multiple taps. This 
process is referred to as filter tap expansion. At step 224, filter coefficients are selected 
according to the multiple class ID which is generated at step 219. At step 225, filtering 
with respect to the deteriorated input signal is executed to generate an undeteriorated 
signal. Filter coefficients are selected adaptively according to the multiple class ID 
which is generated in step 219. 

In one embodiment of the present invention, a three dimensional ADRC process 
may be used to realize spatio-temporal classification, because simple waveform 
classifications such as a two dimensional ADRC process typically cannot structurally 
achieve separation for general motion pictures in the class of FIG. 2B. If both 
stationary and motion areas are processed in the same class ID, error recovery quality is 
degraded because of differences in characteristics of the two areas. 

In another embodiment of the present invention, motion classification, in 
addition to spatial classification, may also be used to provide compact definition of 
temporal characteristics. Further, multiple classification may be added to the classified 
adaptive error recovery method. For example, there are various types of classes, such 
as a motion class, an error class, a spatial activity class and a spatial class explained 
above. The combination of one or more of these different classification methods can 
also improve classification quality. 

FIG. 3 shows an example of motion class tap structures. The example shows 
eight taps in neighborhood of the target error data. In this example, the eight tap 
accumulated temporal difference can be evaluated according to [formula 9], shown 
below, and is classified to four kinds of motion classes by thresholding based on 
[formula 10], shown below. In one embodiment of the present invention, thO is equal 
to 3, thl is equal to 8, and th2 is equal to 24. 



8 



fd = Zk-xM 
1=1 



[formula 9] 



fO (0 < fdKthO) 

1 (thO < fd<th\) 

(thl < fd<th2) 

.3 {thl < fd) 



[formula 10] 
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In the above formulas.yt/ represents an accumulated temporal difference, xj 
represents motion class tap data of the current frame, jc'/ represents the previous frame 
tap data corresponding to the current frame, and mc represents a motion class ID. 
Three thresholds, thO, thl, th2, can be used for this motion classification. 

In one embodiment of the present invention, an error class can be used in 
conjunction with the classified adaptive error recovery method. This classification is 
achieved according to the erroneous data distribution pattern in neighborhood of the 
target data, examples of which are shown in FIG. 4. This example has four error 
classes: an independent error case, a left error case, a right error case, and a three 
consecutive error case. 

Generally speaking, filter coefficients of pixels adjacent to the target data have 
larger weights for error recovery. The data adjacent to the error data has a significant 
impact on the result of error recovery. Error classes can reduce this influence by 
separating different characteristic areas to other classes according to the adjacent 
erroneous data distribution. For the example shown in FIG. 2B, ADRC classification 
generates 16 kinds of ADRC class IDs, where motion and error classification generate 
four kinds of class IDs, respectively. Thus, the number of class IDs equals 16x4x4, or 
256. Classification may be reahzed by representing each signal characteristic. 
Multiple classification can define a suitable class, the class ID, regarding the erroneous 
target data by combining different classification characteristics. 

Adaptive Class Tap Structure 

In one embodiment of the present invention, an adaptive class tap structure can 
be used in conjunction with the classified adaptive error recovery method. FIG. 5 
shows one example of motion class adaptive spatial class tap structures. Intra-frame 
taps can be chosen in a stationary or a slow motion area. Intra-field taps are typically 
used for larger motion areas. Suitable spatial classification is achieved by this adaptive 
processing. 

For example, if intra-frame taps are used for large motion area classification, 
then the generated class distribution may vary widely because of low correlation, and 
therefore it will be difficult to represent the target data characteristics properly. An 
adaptive class tap structure, such as that shown in FIG. 5, is therefore effective. 

Additional examples are shown in FIGs. 6, 7, 8, 9. Spatial class taps are 
typically selected according to a motion and an error class. In addition to the motion 
factor, the erroneous data distribution is taken into account for the spatial class tap 
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definition. The neighboring erroneous data is typically not introduced to the spatial 
classification. By this definition, only valid data is used and the classification accuracy 
is improved. 

Adaptive Filter Tap Structure 

In one embodiment of the present invention, an adaptive filter tap structure 
based on a corresponding class can be used in conjunction with the classified adaptive 
error recovery method. FIG. 10 shows one example of an adaptive filter tap structures 
based on an error class. The filter tap structure regarding the target data is typically 
defined adaptively, preferably avoiding damaged data in neighborhood. Damaged data 
is not chosen for filtering. 

An adaptive filter tap strucmre can be also defined according to motion class, an 
example of which is shown in FIG, 11, In the motion class example shown in FIG, 10, 
motion class 0 corresponds to stationary areas, but motion class 3 corresponds to large 
motion areas. Motion classes 1 and 2 correspond to intermediate motion areas. 

For stationary or quasi-stationary class areas, intra-firame taps are used as shown 
in FIG. 11. At the same time, previous frame data at the target data location may be 
used for error recovery filtering. These areas correspond to motion class 0 and 1. For 
fast motion or moderate motion areas, e;ach filter typically has an intra-field taps 
structure, which is also shown in FIG. 11. As shown by the example in FIG. 11, 
previous frame data is not introduced, and thus weakly correlated data is ignored. 
Filtering quality is typically improved by intra-field taps in such cases. 

FIG. 12 shows an example of motion and error class adaptive filter tap 
structures. FIGs. 10 and 11 represent error and motion class adaptive filter taps, 
respectively. The example shown in FIG. 12 illustrates both adaptive structures with 
error classO, which is the independent error case. Upper adaptive characteristics are 
also shown in this example. In a manner similar to that of FIG. 12, FIG. 13 
corresponds to error class 1, FIG. 14 corresponds to error class 2 and FIG. 15 
corresponds to error class 3. 

Filter Tap Expansion 

In one embodiment of the present invention, filter tap expansion by allocating 
the same coefficient to plural taps can be used in conjunction with the classified 
adaptive error recovery method. Filter tap expansion is also shown by the structures in 
FIGs. 12-15. For example, the filter tap structure has four of the same coefficient taps 
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with motion class 3 in FIG. 12. According to the evaluation results, some tap 
coefficients can be replaced with the same coefficient. The example shown in FIG. 12 
has four W3 coefficients that are allocated at horizontally and vertically symmetric 
locations. By this expansion, 14 coefficients can cover 18 tap areas. This reduction 
method can typically reduce the need for coefficient memory and filtering hardware 
such as adders and multipliers. In one embodiment of the present invention, the 
expansion tap definition may be achieved by evaluation of coefficient distribution and 
visual results. 

Preprocessing For Peripheral Erroneous Data 

In one embodiment of the present invention, preprocessing for peripheral 
erroneous data can be used in conjunction with the classified adaptive error recovery 
method. To achieve error recovery filtering, suitable data is necessary at peripheral 
error locations of filter taps. 

One example of this preprocessing is shown by the flow diagram of FIG. 16. If 
at steps 1601, 1605, or 1609 there is erroneous data at a peripheral location of the target 
data, at steps 1603, 1607, 1611 the erroneous data is replaced with horizontal processed 
data in the case of no horizontal errors. If at steps 1613, 1617, or 1621 there are three 
consecutive horizontal errors, at steps 1615, 1619, or 1623 vertical processing is 
applied for generating preprocessed data. In all erroneous cases around the intra-firame 
data of this example, previous fi-ame data is introduced for error processing, at step 
1625. r 

FIG. 17 shows another preprocessing example that uses a motion adaptive 
process for preprocessing. Using error firee data, motion quantity is detected at the 
motion detection step 1701. Generally speaking, an averaged motion quantity is 
calculated by averaging summed motion quantity with the number of error free data at 
the next step. Motion or stationary taps are chosen at step 1703 according to a 
threshold value of the result of averaged motion quantity. After these steps, processing 
steps 1705 through 1729 are performed in a manner similar to steps 1601 through 1625 
of FIG. 16. The preprocessed data is generated according to these prioritized processes, 
and is introduced for error recovery filtering. 

Spatial Class Reduction 

In one embodiment of the present invention, spatial class reduction can be used 
in conjunction with the classified adaptive error recovery. As explained above, an 
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ADRC class can be used for the spatial classification, given by [formula 3]. This has 
16 kinds of class IDs in the definition of a 4 tap ADRC. These 16 class IDs can be 
reduced to eight kinds of class IDs according to [formula 11], shown below, 

'l2^_l_f2'-'.9, (c>r) [fonnulall] 

I i-1 

where c corresponds to the ADRC class ID, qi is the quantized data and Q is the 
number of quantization bits based on [formula 1] and [formula 2]. 

In one embodiment of the present invention, [formula 11] corresponds to a Ts 
complement operation in binary data of the ADRC code. This is related to the 
symmetric characteristics of each signal waveform. Because ADRC classification is a 
normalization of the target signal waveform, two waveforms which have the relation of 
1 's complement in each ADRC code can be classified in the same class ID. ADRC 
class IDs can typically be halved by this reduction process. An ADRC class reduction 
based on a 4-tap 1-bit ADRC is shown in FIG. 20. In this example, applying [formula 
11] gives eight ADRC class pairs. Each pair contains spatial symmetric patterns, and 
therefore the number of ADRC class IDs can be reduced by half by taking advantage of 
these symmetric patterns. The spatial class reduction technique can also be applied to 
other spatial classification techniques, including but not limited to DPCM and Block 
Truncation Coding (BTC). 

System Structure 

An overall system structure for one embodiment of the present invention, 
including all the processes described above, is shown in FIG. 18. Input data 1801 and 
corresponding error flags 1803 are input to the system. Examining the error flags 1803, 
the input data 1801 is preprocessed at 1805. ADRC classification is performed at 1807, 
motion classification is performed at 1809, and error classification is performed at 
1811. 

In this example, ADRC class taps are chosen adaptively according to the error 
and motion class, such as shown in FIGs. 6, 7, 8, 9. Filter tap data are chosen at 1813 
based on the error and motion class, such as shown in FIGs. 12, 13, 14, 15. Error 
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recovery filtering is performed at 1817 with tap data and filter coefficients selected 
fi-om the coefficient memory 1815 corresponding to the ADRC class ID of 1807, the 
motion class ID of 1809 and the error class ID of 1811. Error recovered data and error 
free input data 1817 are selected at 1821 according to the error flag 1803, which 
produces the output data 1823 of this system. 

FIG. 19 shows an example of coefficient memory contents. It has 4x4x8 or 128 
class IDs according to the multiple classification scheme. Four categories are used for 
an error class, four categories are for a motion class, and eight categories are for an 
ADRC class, which are typically halved according to [formula 11]. Each class 
corresponds to each memory address in FIG. 19. In this example, 14 coefficients are 
stored in each class ID address according to the filter definition, like FIGs. 12, 13, 14, 
15. 

The present invention may be used with any form of correlated data, including 
without limitation photographs or other two-dimensional static images, holograms, or 
other three-dimensional static images, video or other two-dimensional moving images, 
three-dimensional moving images, a monaural sound stream, or sound separated into a 
number of spatially related streams, such as stereo. FIG. 21 shows an example of audio 
signal adaptive classification compatible with the present invention. An example audio 
signal 2101 is monitored at one or more time points tO - t8. The level of the audio 
signal 2101 at time points tO -t8 is given by tap points XO - X8. The dynamic range of 
the audio signal 2101 is given as the difference between the lowest level tap point XO 
and the highest level tap point X4. In case of error recovery for erroneous data at t4, 
multiple classification can be applied with spatial classification like ADRC 
classification and spatial activity classification like dynamic range classification. 
Dynamic range classification is performed by thresholding the dynamic range in a 
manner similar to the motion classification processing of [formula 10]. As described 
above, motion classification, error classification and spatial classification are referred to 
in multiple classification. Spatial activity classification can also be introduced to 
multiple classification for general applications such as video data. In addition to 
dynamic range, the standard deviation, the Laplacian value or the spatial gradient value 
can be introduced for spatial activity classification. 

With the present invention, the quality of data that is recovered due to errors is 
improved by introducing the disclosed technologies to the classified adaptive error 
recovery method. The present invention provides a way to restore a deteriorated signal 
to an undeteriorated signal which minimizes degradations on changing data. 
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While the invention is described in terms of embodiments in a specific system 
environment, those of ordinary skill in the art will recognize that the invention can be 
practiced, with modification, in other and different hardware and software 
environments within the spirit and scope of the appended claims. 
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CLAIMS 

What is claimed is: 

1. A method for restoring a deteriorated input signal comprising the steps of: 
detecting a data point of the deteriorated input signal (1801); 

classifying an area containing the data point with respect to a plurality of class 
types (21 7); 

creating a multiple classification result with the plurahty of class types (219); 
adaptively selecting filter taps (1813) according to the multple classification 
result (221); 

selecting a filter coefficient (1815) according to the multiple classification result 

(224) ; 

creating an undeteriorated data by filtering the data (1817) with the filter 
coefficient (1815) and the filter taps (1813) selected by the multiple classification resuh 

(225) ; and 

outputting an undeteriorated signal (1823) corresponding to the input signal 

(1801). 

2. The method of claim 1 wherein the step of creating a multiple classification 
result comprises creating the multiple classification result based upon a plurality of 
class IDs with the plurality of class types (2 1 9). 

3. The method of claim 1 fiirther comprising the step of adaptively selecting filter 
taps (1813) according to the multple classification result based upon a characteristic of 
an area containing the data point. 

4. The method of claim 1 fiirther comprising the step of adaptively selecting the 
filter taps (1813) based upon characteristics of an area containing the data point. 

5. The method of claim 1 further comprising the step of adaptively selecting the 
filter taps (1813) according to a motion class (1809), 



6. The method of claim 1 further comprising the step of adaptively selecting the 
filter taps (1813) according to an error class (181 1). 
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7. The method of claim 1 further comprising the step of adaptive ly selecting the 
filter taps (1813) according to a motion class (1809) and an error class (1811). 

8. The method of claim 1 further comprising the step of creating an Adaptive 
Dynamic Range Coding (ADRC) class (1807). 

9. The method of claim 1 further comprising the step of creating a Differential 
Pulse Code Modulation (DPCM) class. 

10. The method of claim 1 further comprising the step of creating a Vector 
Quantization (VQ) class. 

1 1. The method of claim 1 further comprising the step of creating a Discrete Cosine 
Transform (DCT) class. 

12. The method of claim 1 further comprising the step of creating a motion class 
(1809). 

13. The method of claim 1 further comprising the step of creating an error class 
(1811). 

14. The method of claim 1 further comprising the step of creating a spatial activity 
class. 

15. The method of claim 1 further comprising the step of creating a dynamic range 
class. 

16. The method of claim 1 further comprising the step of creating a standard 
deviation class. 
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The method of claim 1 further comprising the step of creating a Laplacian class. 
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18. The method of claim 1 further comprising the step of creating a spatial gradient 
class. 

19. A method for restoring a deteriorated input signal comprising the steps of: 
detecting a data point of the deteriorated input signal (1801); 

creating a classification result based upon an area containing the data point 

(217); 

adaptively selecting filter taps (1813) according to the classification result 

(221); 

selecting a filter coefficient (1815) according to the classification result (224); 

creating an undeteriorated data by filtering the data (1817) with the filter 
coefficient (1815) and the filter taps (1813) selected by the classification result (225); 
and 

outputting an undeteriorated signal (1823) corresponding to the input signal 

(1801). 

20. The method of claim 19 wherein the step of creating a classification result 
comprises creating the classification result based upon a plurality of class IDs (219). 

21. The method of claim 19 ftirther comprising the step of adaptively selecting filter 
taps (1813) according to the classification result based upon a characteristic of an area 
containing the data point. 

22. The method of claim 19 further comprising the step of adaptively selecting the 
filter taps (1813) based upon characteristics of an area containing the data point. 

23. The method of claim 19 further comprising the step of adaptively selecting the 
filter taps (1813) according to a motion class (1809). 

24. The method of claim 19 further comprising the step of adaptively selecting the 
filter taps (1813) according to an error class (1811). 

25. The method of claim 19 further comprising the step of creating an Adaptive 
Dynamic Range Coding (ADRC) class (1 807). 
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26. The method of claim 19 further comprising the step of creating a Differential 
Pulse Code Modulation (DPCM) class. 

27. The method of claim 19 further comprising the step of creating a Vector 
Quantization (VQ) class. 

28. The method of claim 19 further comprising the step of creating a Discrete 
Cosine Transform (DCT) class. 

29. The method of claim 19 further comprising the step of creating a motion class 
(1809). 

30. The method of claim 19 further comprising the step of creating an error class 
(1811). 

31. The method of claim 19 further comprising the step of creating a spatial activity 
class. 

32. The method of claim 19 further comprising the step of creating a dynamic range 
class. 

33. The method of claim 19 further comprising the step of creating a standard 
deviation class. 

34. The method of claim 19 further comprising the step of creating a Laplacian 
class. 

35. The method of claim 19 further comprising the step of creating a spatial 
gradient class. 



36. An article of manufacture for use in a computer system to restore a deteriorated 
input signal, the article of manufacture comprising a computer usable medium having 
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computer readable program code means embodied in the medium, the program code 
means including: 

computer readable program code means embodied in the computer usable 
medium for causing a computer to detect a data point of the deteriorated input signal 
(1801); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to classify an area containing the data point with 
respect to a plurality of class types (2 1 7); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to create a multiple classification result with the 
plurality of class types (219); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to adaptively select filter taps (1813) according to the 
multple classification result (221); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to select a filter coefficient (1815) according to the 
multiple classification result (224); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to create an undeteriorated data by filtering the data 
(1817) with the filter coefficient (1815) and the filter taps selected (1813) by the 
multiple classification result (225); and 

computer readable program code means (1821) embodied in the computer 
usable medium for causing a computer to output an undeteriorated signal (1823) 
corresponding to the input signal (1801). 

37, The article of manufacture of claim 36 wherein the computer readable program 
code means embodied in the computer usable medium for causing a computer to create 
a multiple classification result comprises computer readable program code means 
embodied in the computer usable medium for causing a computer to create the multiple 
claissification result based upon a plurality of class IDs with the plurality of class types 
(219). 

38. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
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to adaptively select filter taps (1813) according to the tnultple classification result based 
upon a characteristic of an area containing the data point. 

39. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) based upon characteristics of an area 
containing the data point. 

40. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a 
computer to adaptively select the filter taps (1813) according to a motion class 
(1809). 

41 . The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) according to an error class (181 1). 

42. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) according to a motion class and an error class 
(1809). 

43. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create an Adaptive Dynamic Range Coding (ADRC) class (1 807). 

44. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Differential Pulse Code Modulation (DPCM) class. 

45. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Vector Quantization (VQ) class. 
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46. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Discrete Cosine Transform (DCT) class. 

47. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a motion class (1 809). 

48. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create an error class (1811). 

49. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a spatial activity class. 

50. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a dynamic range class. 

51. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a standard deviation class. 

52. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Laplacian class. 



53. The article of manufacture of claim 36 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a spatial gradient class. 



wo 00/48117 



-24- 



PCT/USOO/03636 



54. An article of manufacture for use in a computer system to restore a deteriorated 
input signal, the article of manufacture comprising a computer usable medium having 
computer readable program code means embodied in the medium, the program code 
means including: 

computer readable program code means embodied in the computer usable 
medium for causing_a computer to detect a data point of the deteriorated input signal 
(1801); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to create a classification result based upon an area 
containing the data point (205); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to adaptively select filter taps (1813) according to the 
classification result (221); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to select a filter coefficient according to the 
classification result (224); 

computer readable program code means embodied in the computer usable 
medium for causing a computer to create an undeteriorated data by filtering the data 
(1817) with the filter coefficient (1815) and the filter taps (1813) selected by the 
classification result (225); and 

computer readable program code means (1821) embodied in the computer 
usable medium for causing a computer to output an undeteriorated signal (1823) 
corresponding to the input signal (1801). 

55. The article of manufacture of claim 54 wherein the computer readable program 
code means embodied in the computer usable medium for causing a computer to create 
a classification result comprises computer readable program code means embodied in 
the computer usable medium for causing a computer to create the classification result 
based upon a plurality of class IDs (219). 

56. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select filter taps (1813) according to the classification result based upon a 
characteristic of an area containing the data point. 
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57. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) based upon characteristics of an area 
containing the data point. 

58. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) according to a motion class (1809). 

59. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to adaptively select the filter taps (1813) according to an error class (181 1). 

60. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create an Adaptive Dynamic Range Coding (ADRC) class (1 807). 

61. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Differential Pulse Code Modulation (DPCM) class. 

62. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Vector Quantization (VQ) class. 

63. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Discrete Cosine Transform (DCT) class. 

64. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a motion class (1 809), 
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program code means embodied in the computer usable medium for causing a computer 
to create an error class (1811). 

66. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a spatial activity class. 

67. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a dynamic range class. 

68. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a standard deviation class. 

69. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a Laplacian class. 

70. The article of manufacture of claim 54 further comprising computer readable 
program code means embodied in the computer usable medium for causing a computer 
to create a spatial gradient class. 

71. An apparatus for restoring a deteriorated input signal comprising: 

a detector to detect a data point of the deteriorated input signal (1801); 

a classifier logically coupled to the detector to classify an area containing the 
data point with respect to a plurality of class types (217); 

a result generator logically coupled to the classifier to create a multiple 
classification result with the plurality of class types (219); 

a first selector logically coupled to the result generator to adaptively select filter 
taps (1813) according to the multple classification result (221) ; 
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a second selector logically coupled to the result generator to select a filter 
coefficient (1815) according to the multiple classification result (224); 

a data restorer logically coupled to the first selector and the second selector to 
create an undeteriorated data by filtering the data (1817) with the filter coefficient 
(1815) and the filter taps (1813) selected by the multiple classification result (225); and 

a transmitter (1821) logically coupled to the data restorer to output an 
undeteriorated signal (1823) corresponding to the input signal (1801). 

72. The apparatus of claim 71 wherein the result generator comprises a result 
generator to create the multiple classification result based upon a plurality of class IDs 
with the plurality of class types. 

73. The apparatus of claim 71 further comprising a third selector to adaptively 
select filter taps according to the multple classification result based upon a 
characteristic of an area containing the data point. 

74. The apparatus of claim 71 further comprising a third selector to adaptively 
select the filter taps (1813) based upon characteristics of an area containing the data 
point 

75. The apparatus of claim 71 further comprising a third selector to adaptively 
select the filter taps (1813) according to a motion class ( 1 809). 

76. The apparatus of claim 71 further comprising a third selector to adaptively 
select the filter taps according to an error class (1811). 

77. The apparatus of claim 71 further comprising a third selector to adaptively 
select the filter taps (1813) according to a motion class (1809) and an error class 
(1811). 



78. The apparatus of claim 71 further comprising a class generator to create an 
Adaptive Dynamic Range Coding (ADRC) class (1807). 
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79. The apparatus of claim 71 further comprising a class generator to create a 
Differential Pulse Code Modulation (DPCM) class, an Adaptive Dynamic Range 
Coding (ADRC) class. 

80. The apparatus of claim 71 further comprising a class generator to create a 
Vector Quantization (VQ) class. 

81. The apparatus of claim 71 further comprising a class generator to create a 
Discrete Cosine Transform (DCT) class. 

82. The apparatus of claim 71 further comprising a class generator to create a 
motion class (1809). 

83. The apparatus of claim 71 further comprising a class generator to create an error 
class (1811). 

84. The apparatus of claim 71 further comprising a class generator to create a 
spatial activity class. 

85. The apparatus of claim 71 further comprising a class generator to create a 
dynamic range class. 

86. The apparatus of claim 71 further comprising a class generator to create a 
standard deviation class. 

87. The apparatus of claim 71 further comprising a class generator to create a 
Laplacian class. 

88. The apparatus of claim 71 further comprising a class generator to create a 
spatial gradient class. 



89. 



An apparatus for restoring a deteriorated input signal comprising: 
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a detector (1805) to detect a data point of the deteriorated input signal (1801); 

a result generator logically coupled to the detector to create a classification 
result based upon an area containing the data point (2 1 7); 

a first selector logically coupled to the result generator to adaptively select filter 
taps (1813) according to the classification result (221); 

a second selector logically coupled to the result generator to select a filter 
coefficient (1815) according to the classification result (224); 

a data restorer (1817) logically coupled to the first selector and the second 
selector to create an undeteriorated data by filtering the data with the filter coefficient 
and the filter taps selected by the classification result (225); and 

a transmitter (1 82 1 ) logically coupled to the data restorer to output an 
undeteriorated signal (1823) corresponding to the input signal (1801). 

90. The apparatus of claim 89 wherein the result generator comprises a result 
generator to create the classification result based upon a plurality of class IDs. 

91. The apparatus of claim 89 further comprising a third selector to adaptively 
select filter taps (1813) according to the classification result based upon a characteristic 
of an area containing the data point. 

92. The apparatus of claim 89 further comprising a third selector to adaptively 
select the filter taps (1813) based upon characteristics of an area containing the data 
point. 

93. The apparatus of claim 89 further comprising a third selector to adaptively 
select the filter taps (1813) according to a motion class ( 1 809). 

94. The apparatus of claim 89 further comprising a third selector to adaptively 
select the filter taps (1813) according to an error class (1811). 

95. The apparatus of claim 89 further comprising a class generator to create an 
Adaptive Dynamic Range Coding (ADRC) class (1807). 
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96. The apparatus of claim 89 further comprising a class generator to create a 
Differential Pulse Code Modulation (DPCM) class, an Adaptive Dynamic Range 
Coding (ADRC) class. 

97. The apparatus of claim 89 further comprising a class generator to create a 
Vector Quantization-(VQ) class. 

98. The apparatus of claim 89 further comprising a class generator to create a 
Discrete Cosine Transform (DCT) class. 

99. The apparatus of claim 89 further comprising a class generator to create a 
motion class (1 809). 

100. The apparatus of claim 89 further comprising a class generator to create an error 
class (1811). 

101. The apparatus of claim 89 further comprising a class generator to create a 
spatial activity class. 

102. The apparatus of claim 89 further comprising a class generator to create a 
dynamic range class. 

103. The apparatus of claim 89 further comprising a class generator to create a 
standard deviation class. 

104. The apparatus of claim 89 further comprising a class generator to create a 
Laplacian class. 

105. The apparatus of claim 89 further comprising a class generator to create a 
spatial gradient class. 



106. 



An apparatus for restoring a deteriorated input signal comprising: 
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means for detecting a data point of the deteriorated input signal ( 1 80 1 ); 

means for classifying an area containing the data point with respect to a 
plurality of class types (217); 

means for creating a multiple classification result with the plurality of class 
types (219); 

means for adaptively selecting filter taps (1813) according to the multple 
classification result (221); 

means for selecting a filter coefficient (1815) according to the multiple 
classification result (224); 

means for creating an undeteriorated data by filtering the data (1817) with the 
filter coefficient (1815) and the filter taps (1813) selected by the multiple classification 
result (225); and 

means for outputting (1821) an undeteriorated signal (1823) corresponding to 
the input signal (1801). 



107. An apparatus for restoring a deteriorated input signal comprising: 

means for detecting a data point of the deteriorated input signal (1801); 
means for creating a classification result based upon an area containing the data 

point (217); 

means for adaptively selecting filter taps (1813) according to the classification 
result (221); 

means for selecting a filter coefficient (1815) according to the classification 
result (224); 

means for creating an undeteriorated data by filtering the data (1817) with the 
filter coefficient (1815) and the filter taps (1813) selected by the classification result 
(225); and 

means for outputting (1821) an undeteriorated signal (1823) corresponding to 
the input signal (1801). 
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